INVITED REVIEW Predictive Reward Signal of Dopamine Neurons

Schultz, Wolfram. Predictive reward signal of dopamine neurons. is called rewards, which elicit and reinforce approach behavJ. Neurophysiol. 80: 1–27, 1998. The effects of lesions, receptor ior. The functions of rewards were developed further during blocking, electrical self-stimulation, and drugs of abuse suggest the evolution of higher mammals to support more sophistithat midbrain dopamine systems are involved in processing reward cated forms of individual and social behavior. Thus biologiinformation and learning approach behavior. Most dopamine neucal and cognitive needs define the nature of rewards, and rons show phasic activations after primary liquid and food rewards the availability of rewards determines some of the basic and conditioned, reward-predicting visual and auditory stimuli. parameters of the subject’s life conditions. They show biphasic, activation-depression responses after stimuli Rewards come in various physical forms, are highly variable that resemble reward-predicting stimuli or are novel or particularly in time and depend on the particular environment of the subject. salient. However, only few phasic activations follow aversive stimuli. Thus dopamine neurons label environmental stimuli with appeDespite their importance, rewards do not influence the brain titive value, predict and detect rewards and signal alerting and through dedicated peripheral receptors tuned to a limited range motivating events. By failing to discriminate between different of physical modalities as is the case for primary sensory sysrewards, dopamine neurons appear to emit an alerting message tems. Rather, reward information is extracted by the brain from about the surprising presence or absence of rewards. All responses a large variety of polysensory, inhomogeneous, and inconstant to rewards and reward-predicting stimuli depend on event predictstimuli by using particular neuronal mechanisms. The highly ability. Dopamine neurons are activated by rewarding events that variable nature of rewards requires high degrees of adaptation are better than predicted, remain uninfluenced by events that are in neuronal systems processing them. as good as predicted, and are depressed by events that are worse One of the principal neuronal systems involved in prothan predicted. By signaling rewards according to a prediction cessing reward information appears to be the dopamine syserror, dopamine responses have the formal characteristics of a teaching signal postulated by reinforcement learning theories. Dotem. Behavioral studies show that dopamine projections to pamine responses transfer during learning from primary rewards the striatum and frontal cortex play a central role in mediatto reward-predicting stimuli. This may contribute to neuronal ing the effects of rewards on approach behavior and learning. mechanisms underlying the retrograde action of rewards, one of These results are derived from selective lesions of different the main puzzles in reinforcement learning. The impulse response components of dopamine systems, systemic and intracerebral releases a short pulse of dopamine onto many dendrites, thus broadadministration of direct and indirect dopamine receptor agocasting a rather global reinforcement signal to postsynaptic neunist and antagonist drugs, electrical self-stimulation, and rons. This signal may improve approach behavior by providing self-administration of major drugs of abuse, such as cocaine, advance reward information before the behavior occurs, and may amphetamine, opiates, alcohol, and nicotine (Beninger and contribute to learning by modifying synaptic transmission. The Hahn 1983; Di Chiara 1995; Fibiger and Phillips 1986; Robdopamine reward signal is supplemented by activity in neurons in striatum, frontal cortex, and amygdala, which process specific rebins and Everitt 1992; Robinson and Berridge 1993; Wise ward information but do not emit a global reward prediction error 1996; Wise and Hoffman 1992; Wise et al. 1978). signal. A cooperation between the different reward signals may The present article summarizes recent research concerning assure the use of specific rewards for selectively reinforcing behavthe signaling of environmental motivating stimuli by dopaiors. Among the other projection systems, noradrenaline neurons mine neurons and evaluates the potential functions of these predominantly serve attentional mechanisms and nucleus basalis signals for modifying behavioral reactions by reference to neurons code rewards heterogeneously. Cerebellar climbing fibers anatomic organization, learning theories, artificial neuronal signal errors in motor performance or errors in the prediction of models, other neuronal systems, and deficits after lesions. aversive events to cerebellar Purkinje cells. Most deficits following All known response characteristics of dopamine neurons will dopamine-depleting lesions are not easily explained by a defective be described, but predominantly the responses to rewardreward signal but may reflect the absence of a general enabling function of tonic levels of extracellular dopamine. Thus dopamine related stimuli will be conceptualized because they are the systems may have two functions, the phasic transmission of reward best understood presently. Because of the large amount of information and the tonic enabling of postsynaptic neurons. data available in the literature, the principal system discussed will be the nigrostriatal dopamine projection, but projections from midbrain dopamine neurons to ventral striatum and I N T R O D U C T I O N frontal cortex also will be considered as far as the present knowledge allows. When multicellular organisms arose through the evolution of self-reproducing molecules, they developed endogenous, R E W A R D S A N D P R E D I C T I O N S autoregulatory mechanisms assuring that their needs for welFunctions of rewards fare and survival were met. Subjects engage in various forms of approach behavior to obtain resources for maintaining Certain objects and events in the environment are of particular motivational significance by their effects on welfare, homeostatic balance and to reproduce. One class of resources